Click any tag below to further narrow down your results
Links
Hex built a suite of analytical evals to test data-analysis models and found Claude Fable 5 outperforms its Opus 4.x predecessors by 10–15%, nailing both semantically modeled and raw-data tasks with fewer mistakes. They’ve also designed a tougher “Frontier” benchmark for long-horizon, open-ended scenarios, where Fable 5’s careful assumptions and cross-checks boost its pass rate to around 58%.
- Claude Fable 5 beats Opus 4.7 by 10-15 points on Hex's core benchmarks, scoring 93%+ on Analytical Hard/Semantically Modeled tests and 65% on Semantically Unmodeled tasks, versus prior Opus versions' single-digit gains
- Fable's advantage comes from following a "golden workflow" (starting in the semantic layer, cross-checking raw SQL) and transparently stating assumptions, which lets it catch errors like a cents-for-dollars mistake that Opus misses
- On Hex's new "Frontier" benchmark for long-horizon, open-ended tasks, Fable at Max Effort hits 58% pass rate, notably outperforming other setups
Simon Willison runs Claude Fable 5 through its paces, finding it slower and pricier than Opus 4.8 but far more knowledgeable thanks to its 1 million-token context. He tests it on real-world coding tasks—upgrading a MicroPython sandbox to full CPython in WASM and adding pause-resume hooks to Datasette Agent—showing it can build complex features end-to-end.
- Fable 5 costs twice as much as Opus 4.5–4.8 ($10/$50 per million tokens vs their pricing) and runs slower, but handles every complex prompt thrown at it.
- Its 1M-token context gives it dramatically deeper recall than Opus 4.8—listing a dozen detailed open-source projects with dates versus Opus's brief list of four.
- It successfully converted a MicroPython-wasm project to full CPython in WASM, ultimately producing a working 13.9 MB wheel runnable via uv, demonstrating real end-to-end coding capability.